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ABSTRACT 



The success of alternative assessments such as the Vermont 
Assessment Program (VAP) is heavily dependent on the involvement and 
commitment of teachers. This paper focuses on the implementation of the 
writing portfolio component of the VAP in the classrooms of four teachers who 
have different knowledge and beliefs about teaching and assessment. The 
historical context and basic structure of the writing__porLtion__o±_.the^_-VAP~-are— 

described. Vermont schools, which are predomii. i.z l^~rurai"; have a long 

tradition of local governance, teacher autonomy, and inclusion of students 
with disabilities in the regular classroom. Four experienced fifth-grade 
teachers, first-time implementors of the VAP, were studied to determine the 
effects of this policy on teachers in different contexts. These four teachers 
represent four different belief cultures: curriculum-oriented, 
process- inquiry, polytheoretic , and minimalist. There were varying degrees 
and types of influences in these four classrooms as a result of their first 
year of participation in the portfolio system. The teachers began with 
different beliefs and practices, approached the VAP task differently, and 
finished with different degrees of involvement at the end of the year. The 
influence of the VAP in the classrooms ranged from none to changes in 
organizational procedures, to students writing more and producing more kinds 
of writing, and to students engaging in a new process of reflection and 
self-evaluation. Results suggest the need for long-range, flexible, and 
multidimensional approaches to professional development and the 
implementation of alternative assessment. An appendix provides an analytical 
writing assessment guide. (Contains 4 figures and 18 references.) (SLD) 
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Individual Action and Reflection: 

Four Case Studies of Teachers' Responses to a Statewide Assessment Policy 

Patricia A. Daniels 
University of Vermont 



I do think the VAP is exciting. I think it probably needs some wrinkles 
ironed out, but everything does when it’s new. But 1 think that if those 
who are involved in developing it are responsive to the people on the ground 
floor and using it, and will listen, some very positive things will come of it. 

Linda, fifth grade teacher 

Linda's reflection on her first year of participation in the Vermont 
Assessment Program (VAP) highlights the rationale for these case studies: it is 
important to understand the particular experiences of the teachers as 
implementors of educational reform. The success of alternative assessments such 
as the Vermont Assessment Program is heavily dependent on the involvement 
and commitment of teachers (Abruscato, 1993; Hewitt, 1995; Worthen, 1993). 
From the beginning, Vermont teachers have been the lifeblood of the VAP. In 
partnership with Department of Education specialists, teachers initially 
developed and piloted the VAP system. Within that system, they remain 
indispensable as "network leaders" who facilitate, support and inform colleagues, 
as evaluators of local portfolios, and as scorers of state-selected portfolios. Most 
importantly, however, teachers are the key decision-makers about how portfolios 
are used and constructed in classrooms. In effect, they are the "gatekeeper(s) of 
alternative assessment (Worthen, 1993)." It is important to investigate their 
experiences and listen to their perspectives so that policy makers can understand 
and respond to the nature of the impact of the VAP initiative on its primary 
stakeholders: teachers and students. 
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This paper focuses on the implementation of the writing portfolio 
component Of the Vermont Assessment Program in the classrooms of four 
teachers who hold diverse knowledge and beliefs about teaching and assessment. 
First, I summarize the historical context and basic structure of the writing portion 
of the VAP. Then I outline the case study methodology. Finally, I introduce four 
fifth grade teachers who are first-time implementors of the VAP. Using a cross- 
case analysis of four in-depth case studies of these teachers, I explore to what 
extent a single policy is likely to have a common effect on teachers from a variety 
of contexts who espouse different beliefs and practices. I describe and compare 
each teacher's approach to teaching writing, the portfolio development process in 
each classroom, and their individual reflections about the impact of the VAP. In 
conclusion, I address implications for policy makers and professional 
development. 

Background Information 
A Portfolio Culture in Vermont 

Vermont sets a unique context for educational reform. It is one of the 
most rural states in the country. Its size, (9,614 square miles) is comparable to 
that of a large county in most of our larger states. It has a total population of 
only 580,209 people in the latest census, and a total k-12 public school population 
of 103,317 students. Most of its elementary schools are small and rural; over 50% 
have only one classroom at each grade level, and multi-aged groupings are 
common. Vermont schools have a long tradition that values local governance, 
teacher autonomy, and full inclusion of students with disabilities in regular 
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During the past decade, the term "portfolios" became embedded in the 
culture of Vermont schools. During the 1980's, Vermont teachers (myself 
included) flocked to statewide courses and workshops to learn about "The 
Vermont Writing Process" (Hewitt, 1995). Grounded in the philosophy of 
Donald Graves (1987), Lucy Calkins (1986), and Nancie Atwell (1987), this 
workshop approach to teaching writing first engaged teachers and 
administrators in the writing process that they would use with their students. 
Many Vermont classrooms transformed from a traditional approach where 
writing was assigned and taught as a static product, to one in which writing 
became an ongoing process that engaged students and teachers in discussing 
their writing with each other through stages that included prewriting, drafting, 
revising, editing, and sometimes publishing a variety of pieces. "Portfolios" were 
typically the folders in which students kept their writing at various stages of the 
process. 

Vermont had no uniform statewide assessment program before 1991. 
From 1988 to 1991, in response to a legislative prod for accountability, leaders 
from the Department of Education, classroom teachers, other educators, and lay 
people gathered to develop a statewide approach to assessment. Already 
grounded in process writing and portfolios as collections, these educators 
decided that traditional standardized tests did not provide the kind of 
information the state needed about what students know and can do, and also 
did not push student learning in the direction it should go (Hewitt, 1995). Using 
extensive teacher input, Vermonters created their own revolutionary alternative 
portfolio-based assessment system for writing and mathematics, which they 
piloted during the 1990-1991 school year. 
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Thus, Vermont became the first state in the nation to implement a 
statewide portfolio-based assessment program (Hewitt, 1995). Although the 
VAP was never mandated, from 1991-1994, VAP portfolios for both math and 
writing were developed by students in nearly all Vermont fourth and eighth 
grade classrooms. In September, 1995, the Department of Education moved the 
writing assessment to the fifth grade. This decision was in response to concerns 
raised by fourth grade teachers and school administrators about the stress on 
fourth grade teachers and students, implementing both writing and math 
portfolios within the same classroom. Another implied rationale was to promote 
change in the instruction and assessment of writing at grades other than four and 
eight (Vermont Department of Education, March/ April, 1994). Fifth grade 
teachers who were in their first year of implementing writing portfolios for the 
VAP in 1995-1996, therefore, worked with students who had developed the VAP 
portfolios during their previous year in fourth grade. 

The VAP Writing Assessment 

There are two required components of the VAP: portfolios of selected 
students' work and a "uniform test" administered to all fifth and eighth grade 
students. Teachers are encouraged to develop portfolios with all students, and to 
score their own students' portfolios and best pieces. Ideally, writing for 
portfolios is collected, reflected upon, and assessed by all fifth and eighth gr^de 
students throughout the school year and maintained in a working folder or 
portfolio, as possible pieces to be included in the VAP portfolio. Not all 
portfolios are sent to the state for scoring, however. In May, the Department of 
Education requests a randomly-selected sample of portfolios from students in 
each supervisory union to be sent to a central location and scored by teams of 
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teachers. These writing portfolios are structured to contain six various genre of 
writing from that academic year, including a student-selected "best piece" which 
the student feels represents h/her best work, and a written reflection letter about 
reasons for choosing that piece. Teacher-scorers apply a four-point rubric 
(extensively, frequently, sometimes, rarely) against specific performance criteria 
in five dimensions: purpose, organization, details, voice /tone, grammar /usage/ 
mechanics (see Appendix). In contrast, the Uniform Test of Writing is a direct 
writing assessment using a single prompt that is administered and collected from 
all Vermont students in grades five and eight at the same time as the portfolios, 
and scored by an outside agency using the same criteria as used with the 
portfolios. 

Methodology 

Data Sources 

These case studies are grounded in rich and varied quantitative and 
qualitative data: (1) surveys, (2) indepth pre-and post-implemenation interviews 
with participant teachers, (3) interviews with influential others (fourth grade 
teachers, principals, and network leaders), (4) detailed, audio taped classroom 
observations, and (4) portfolios and other relevant documents. A team of six 
university researchers collected these data during a one year period, from 
August, 1995-July, 1996. During frequent team meetings, we discussed emerging 
trends, affirmed perceptions, and developed interpretations. 

Participant Selection 

Surveys . Because teachers teach differently, based on their beliefs and 
orientations towards teaching and learning (Lipson & Goldhaber, 1993), we 
selected teachers who represent maximum philosophical diversity in regards to 
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teaching and learning. We initially surveyed all of Vermont’s fifth grade 
teachers. Built into the survey (Lipson & Goldhaber, 1993) were items that 
allowed us to look at teacher beliefs from four different perspectives. Two of 
these focused on views of learning (behaviorist and interactive) and two focused 
on views of teaching (child-centered and systems oriented). A subsequent 
cluster analysis of respondents resulted in four identifiable groups of teachers: 
Curriculum-oriented, Process-Inquiry, Polytheoretic, and Minimalist (Lipson, 
1995a). Teachers' membership in these clusters was strongly associated with 
their self-identified instructional practices and their orientation to assessment, as 
shown in their individual survey responses. 

Twelve teachers were selected for indepth inquiry. Of these twelve, four 
are represented in this paper — one from each of the belief clusters. (Pseudonyms 
are used.) In general terms, the clusters compare as shown below (Lipson, 
1995a): 

Figure 1 

Summary of Teacher Belief Clusters 

Group % of Respondents Sample of beliefs 



Curriculum-Oriented (Maura) 41% 

High scores on Behaviorist and Systems 
scales. Lowest scores on Interactionist 
and Child-Centered scales 



-Parents and others need 
information that compares 
students. 

-VAP a poor match to present 
instruction. 



Process-Inquiry (Eve) 19% 

Hi ghest scores on Interactionist and 
Child-centered scales. High Systems 
scores also. 



-Children, parents and teachers 
should be equal participants in 
collecting, selecting and evalu- 
ating information. 

-VAP a good match to present 
instruction 
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Polytheoretic (Linda) 21% 

Very high scores on all scales. 

Highest scores on Behaviorist and 
Systems, but also very high on other two. 

Minimalist (Leslie) 18% 

Low to moderate scores on all four 
scales, seeming uncertain about 
adopting any perspective very 
strongly. Lowest scores on System 
scale. 



-Claimed multiple beliefs and 
often competing practices. 



-Seem to hold no clear theoreti- 
cal perspective on teaching and 
learning. 



To further maximize diversity, we selected teachers from varied contexts 
within different geographic regions of Vermont. Figure 2 displays variety 
across the contexts of the four teachers. 



Name 

Maura 



Eve 



Linda 



Leslie 



Figure 2 



Summary of Teachers’ Background and Contexts 

Gr oup Experien ce C o n text. 



Curriculum 



Process-Inquiry 



Polytheoretic 



Minimalist 



20 years K-8 small town elementary school 

M.Ed. Five 5th grades 

Low-middle SES 
23 students 

High special needs student pop. 

21 years 5-8 suburban Middle School 

M.Ed. Four fifth grades 

Low-high SES 
50 students (team teaches) 



26 years k-8 rural elementary school 

BA One 5th grade 

low SES 

18 students (14 girls, 4 boys) 

12 years k-6 suburban elementary school 

BA Three 5th grades 

low-middle SES 
20 students 
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Maura, a curriculum-oriented teacher, and Linda, a polytheoretic teacher, 
both teach in schools that house students from K-8. but their schools vary 
significantly in size. One is a rural school with a single fifth grade, while the 
other is a small town school with five fifth grades. Leslie, a minimalist teacher, 
and Eve, a process inquiry teacher, both teach in suburban schools that are 
similar in size, but one is a 5-8 middle school, and the other a k-6 elementary 
school. Eve works closely in a team-teaching situation, whereas, the other three 
have largely self-contained classrooms. Linda's 18 students during the year of 
the study were skewed in gender. Maura's class had an unusually high ratio of 
special needs students. 

There are also factors common to all four teachers. First, they are all 
"experienced" teachers. Maura, Eve and Linda each taught elementary school for 
over 20 years prior to our study. Although Leslie is the least experienced of the 
four, she had taught for 12 years, and is considered "experienced." Interviews 
with administrators and 4th grade teachers at each school show that all four 
participants are highly regarded by their administrators and peers. In addition, 
our observations and documents, as well as teacher self-reports, show that each 
teacher used a type of "process writing" and that there was an impressive 
volume of writing produced by students in each classroom. 

Observational data and related documents. The four teachers were 
observed for eight full days (four two-day observations, during the months of 
November, January, April and May). Field notes for each case included a 
detailed written running record of classroom activity, with times recorded every 
3-5 minutes. During these visits, the teacher wore a wireless microphone so that 
all teacher talk could be recorded. The taped records were used to support and 
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clarify our written records. Documents connected to each observation included 
photographs, representative examples of student work, teachers' outlines of 
written daily plans, portfolios constructed at different phases in each classroom, 
and completed VAP portfolios. 

Interviews . Lengthy pre-and post-implementation interviews were 
conducted with each teacher, focusing on classroom instruction and assessment 
practices, use of portfolios, and perceptions of the VAP. Additional brief 
interviews were conducted before and after each two day observation to clarify 
and to add the teacher's perspective to what was observed. Within each 
observation were instances of impromptu, informal conversations we called 
"teacher talk". Some of these were taped and transcribed verbatim; others were 
reconstructed as accurately as possible. Additional interviews were conducted 
with the school principal, a 4th grade teacher in the same school or district, and 
the area VAP network leader. These created a broader local context for 
understanding the variety of influences on the teacher's assessment and 
instructional practices. 

Data Analvsis 

7 

Shortly after the fourth two-day observation, each observer wrote a 
summary description of what was observed during the year. This was read by 
each teacher during the final interview. All teachers agreed with the 
characterizations in these summaries, although specific information was revised 
in two, based on teachers' responses. 

These data were arranged chronologically in individual case books for 
each teacher, which include the "running records" of eight days of observations, 
documents that surround these observations, all transcribed interviews with that 
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teacher, observers' summaries of reactions/highlights from each visit, and 
examples of representative student work and portfolios constructed in each 
classroom. Primary patterns were identified, coded and categorized, using a 
content analysis procedure (Ely, 1991; Glesne & Peshkin, 1992; Patton, 1990). In 
doing so, we triangulated our data sources. Interview data were examined for 
consistency with direct observations, and documents were analyzed for 
additional insights into the characterization of assessment and instructional 
practices. Individual case studies were written from these data. 

For cross-case analysis: I used four individual completed case studies of 
one teacher from each belief cluster (Daniels, 1995; 1996; Lipson, 1995b; 1996) 
and the associated summary descriptions. First, I created a meta matrix: 

Figure 3 

Methodology: Format of Meta-matrix 



Teacher & 
Group 


What teacher 
says about self 


Typical writing 
period 


How VAP 
portfolios were 
constructed 


Teacher's 
perspective of the 
VAP 


Maura 

Curriculum- 

oriented 










Leslie 

Minimalist 










Eve 

Process Inquiry 




- . 






Linda 

Polytheoretic 











It displayed four columns of information, using the same framework used for the 
data analysis in each of the case studies: (1) what the teacher says about 
h/herself, (2) what happens during a typical writing period in that classroom. 
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(3) how the VAP portfolios were constructed, and (4) the teacher's individual 
perspective on the VAP experience. Reading across the rows revealed a sense of 
individual teacher's experiences, and reading down the columns provided a 
cross-case comparison of data in the four categories. 

From this process, similarities and differences among cases became 
visible. The large amount of data was progressively clustered and refined into 
quotes and summarizing phrases on a second matrix with identical categories. 
Salient information from this display was then condensed to three categories that 
address this study's general question of to what extent a single policy is likely to 
have a common effect on teachers: What was the focus of writing instruction in 

each classroom throughout the year? To what extent did the VAP portfolios 
become central to classroom instruction? Did the teacher report an impact from 
the VAP? If so, what was the nature of the impact? 



Figure 4 displays the nature of our findings in three categories: (1) 
whether classroom writing instruction focused on content of students' writing 
(rather than editing), (2) whether the development of VAP portfolios became central 
to writing instruction in each classroom, and (3) whether teachers reported an 
impact of the V AP on their writing instruction. Next, I present the data 
represented by each column. 



Findings 
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Figure 4 

Individual Action and Reflection: Teachers' Responses to the Vermont Assessment Program 



Teacher &c 
Group Membership 


Instructional focus on 
content of students' 
writing? 


Centrality of VAP 
portfolios to 
classroom instruction? 


Teacher-reported 
impact of VAP? 


Maura 

Curriculum- 

oriented 


- 


- 


+ 


Leslie 

Minimalist 


- 


- 


- 


Eve 

Process-Inquiry 


+ 


- 


+ 


Linda 

Polytheoretic 


+ 


+ 


+ 



Teaching Writing 

All four teachers said that they used the "process writing" approach, 
guiding their students through stages of pre-writing, revising and publishing. 
And we did, indeed, observe these writing stages during typical writing periods 
in each of these classrooms. What was done with students in the name of 
"process writing," however, varied markedly. 

Maura, a curriculum-oriented teacher, taught writing in a very organized, 
methodical way. This is not surprising, given that "curriculum-oriented" 
teachers generally teach skills in sequential order and base judgments about 
students on the completion of learning outcomes. Writing happened as one part 
of a daily 45 minute Language Arts period, during which spelling and language 
arts subskills were taught separately. Students worked individually on teacher- 
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assigned topics, creating several revisions, in order to produce a perfect end 
product for each piece they started. This approach to teaching writing felt like 
"hard work" to Maura: 

It's hard for me to teach writing because it is a slow process. ...I have 
to be careful that when we get to our first, second, third, fourth and 
fifth drafts that we haven't lost the whole meaning of why were doing it 
in the first place. 

While students wrote, Maura circulated from desk to desk, monitoring 
whether students were on task, providing brief, positive comments ("Good job," 
"Nice work"), and editing papers for spelling and mechanics. Comments about 
the actual content of the writing were sparse, and very brief. In addition, there 
were no peer conferences. Maura felt that the high ratio of special needs students 
in her class made peer coferencing difficult and ineffective. 

Teachers from the "minimalist" belief cluster are least likely to report that 
they provide a sustained writing period. Leslie's students, however, had a 45 
minute block for Writer's Workshop daily, distinguishing her from other 
minimalist teachers. During this time, her students wrote about a variety of self- 
selected and teacher-assigned topics. In addition, students produced writing 
across curricular areas, using response journals, learning logs and projects. She 
believed that "the more they write, the better they're going to get at it." 

The nature of the feedback she gave to students about their writing mirrored 
Maura's. She monitored their writing to provide encouragement and to keep 
them on task, but did not give specific feedback about the content. When 
students shared what they wrote, she responded briefly, "Nice work," or "That's a 
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good start," and moved on to the next student. Students shared writing with 
each other, but gave the same type of brief, non-specific feedback. 

Eve, a process-inquiry teacher, and Linda, a polytheoretic, taught process 
writing very differently than Maura and Leslie. This is not surprising, given that 
teachers in these two groups had high interactionist scores on the surveys of 
beliefs and practices, compared to the low scores from the minimalists and 
curricularists. In Eve's and Linda's classrooms, writing occurred throughout the 
day, but direct writing instruction happened during a long block of integrated 
reading/language arts time each morning. The process and content of students' 
writing was the focus. Both of these teachers encouraged students to make 
personal connections to their writing. Eve felt that writing should be "authentic" 
and that 

It's got to come from inside the writer. It's got to be a reflection of the 
thought process that belongs to that person. If I were to order the criteria 
the state has put forth, I would say voice and tone has got to be right at 
the top. ...I think a good piece of writing has to show the soul. 

Linda echoed this belief. She liked writing that focused on personal meaning 
and helped her to get to know her students better. She stated, "What's exciting is 
seeing what's inside the students. Sometimes, it's incredible, the depth of feeling 
that's in there." 

However similar their basic beliefs, Eve and Linda were at very different 
points in their writing programs during their first year of implementing the VAP. 
Eve had a well-established writing program. From the beginning of the year, she 
gave very straightforward feedback about the quality of the students' writing, 
focusing on word usage, delivery, and effective imagery. Student self-evaluation 
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and peer conferences were ongoing. Responses that students offered to each 
other when sharing their writing were specifically focused on language and 
content. She incorporated the language of the VAP criteria in her mini-lessons 
and student conferences. On the other hand, Linda's year was characterized by a 
significant shift in the focus of her writing instruction, as well as in the nature of 
the responses she gave students about their writing. Early in the year, her 
specific feedback to students focused on mechanics and organization. 
Increasingly, she centered her whole group lessons on specific components of the 
VAP criteria, especially voice and tone, and focused student efforts on making 
writing personally meaningful. Whole class conferences became an integral part 
of each day, during which she and her students freely shared what they wrote, 
eliciting responses from each other about the language and content of their 
writing. 

Doing the VAP: Centrality to Instruction 

Portfolios as product or process . Two dimensions need to be considered 
when considering the centrality of the statewide assessment program on practice: 
the portfolio as a product and process and the classroom use of the evaluative 
criteria used for statewide assessment of portfolios. In all four classrooms, 
teachers created special "portfolio" folders for the products of VAP work, but 
these portfolios functioned differently. 

In Maura's, Eve's and Leslie's classrooms, the portfolio folders were 
repositories for finished work, and not integral to ongoing writing instruction 
and assessment. Students used a different folder or notebook for current writing 
projects. The VAP portfolios never became central to the actual writing program 
in the classroom. They were kept by the teacher, and students seldom visited 




ERIC 



16 



them. Maura kept her students' portfolios by her desk and focused her efforts 
immediately on saving "every little scrap," being organized in order to fulfill the 
VAP requirements. Her students' portfolio folders were bulging by January. 
Leslie and Eve, however, did not collect writing in the folders that they created 
for the VAP initially, although masses of student writing was produced in each 
room. Consequently, when it was time to prepare VAP portfolios, they wished 
they had saved more student work, and resolved to do so the next year. 

In Linda's classroom, however, the portfolio folders were constantly in 
use and contained writing in all stages of the process. Early on, she talked with 
the students about the Vermont Assessment Program. Linda and her students 
decided together to experiment with larger blocks of time for writing on a daily 
basis. In addition, entire Friday mornings were called "Portfolio" and were given 
completely to writing. Students were aware throughout the year, that anything 
they wrote was a potential VAP portfolio document. They voiced opinions about 
"doing portfolios" and wrote about its pros and cons in their journals. They 
understood that the VAP would require them to have several kinds of writing, 
and bemoaned the fact that their valued poetry could not be scored. 

As noted in the previous section, in Eve's and Linda's classrooms there 
was a second kind of impact--the components of the VAP assessment criteria 
were the focus of instructional mini- lessons and conversations about writing. 

Eve alone, however, used it for her own assessment purposes and taught her 
students to use the criteria to evaluate writing. This was a refinement of her 
practice, which already included ongoing writing, conferencing and reflecting by 
students. 
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Final steps in constructing YAP portfolios. In May, five or six students 
were randomly selected from each classroom to submit portfolios for evaluation 
by the state. The final steps taken to construct portfolios for the VAP varied in 
each classroom. In Eve’s, Leslie's, and Linda's classrooms, all students created 
portfolios in the same way, as if their name had been selected by the state. 

Maura, however, had a dramatically different approach than the other three. She 
and her classroom aide referred to the students whose portfolios were selected as 
"the chosen few" and she worked with them separately. While her aide worked 
with the rest of the class on a different assignment, Maura took these students 
aside and insured that they had neatly-finished pieces for each category and a 
complete table of contents, guiding them through each step. After these 
students' portfolios were completed, the remainder of the students quickly 
constructed portfolios, knowing that theirs was not going to the state. Although 
Maura stands alone in this group, examination of our remaining eight cases 
suggests that other teachers made special arrangements. 

Eve and Linda gave all students guidelines on how to select a "best piece" 
in each category, and instructed them to use the language from the criteria in 
writing their letter to the portfolio reader. Maura gave no particular instruction 
about how to choose pieces for the portfolio, except to "pick quality work." 

Leslie asked all students to address several questions in the letter to the reader, 
and helped them to pick out their best pieces. In all four classrooms, however, 
student choices were ultimately honored, whether or not the teacher agreed. 

Impact of the VAP: What Teachers Say 

Teachers' self-reports affirm our observation information about the nature 
of the impact in each classroom. For the most part, these teachers were clear 
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about the influences of the state's assessment policy on their practice, and about 
what they would do differently during a subsequent year. 

Leslie was the exception. We observed no change in her writing 
instruction as a result of the VAP. She also reported a minimal impact: "It gets 
them writing more" -- in a classroom where students already produced large 
amounts of writing. She stated no plans for changing her writing instruction as 
a result of the VAP. She did, however, plan to save more writing during the 
second year. 

Eve reported two impacts she considered significant: finishing more 
writing and providing criteria for instruction and assessment of writing. 

It's given me direction. They finished far more writing this year than 
ever because of the portfolio... I've gone from grading everything to 
focusing on one piece. I've got to thank the portfolio for saying that these 
are the criteria that we should be looking for in a piece, and to be able to 
work one at a time with a piece of writing... 

She was also unequivocally positive about the benefits of the criteria as a way to 
help her provide useful feedback to students about their writing. Like Leslie, 
however, she stated no plans for changing future writing instruction, other than 
saving more writing sooner. 

In Maura's room, the observable change was that student writing was 
saved for the first time. Maura also reports that the VAP had a significant and 
positive impact that we did not observe: 

It does make you think about how you teach writing, the best 
way to teach writing to get the results you want. Children need 
to know how to write, and I think starting with the rubric the way 
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they have it, that's a good place to start. Now I see the reason. I see 
the need. I think the state is doing a good thing. 

Maura announced her plans to use the VAP criteria with her students the next 
year. In a telephone conversation the following October, Maura confirmed that 
she had done this, and that her students were, consequently, "writing wonderful 
things." 

Linda reported the impact in very different terms. While explaining a 
change in the focus of her writing instruction from mechanics to meaning, she 
talked more about the impact it had on her students: 

I think I've empowered them all. That grows as the year goes on. In 
the fall, a lot comes from me. It is a lot more directive. But as they 
pick up the skills, my role kind of fades. It's still there, but it's un- 
obtrusive, less directive, and more of a guide, less of a director..It's 
much more obvious this year. The students plan a major role in 
critiquing their work. ...The growth that I've seen as far as the writing 
that's produced! Also the growth in ability to look at a piece and critique 
it and see what makes it special and what needs to be done to make it 
special — It's phenomenal! 

For the next year, Linda planned to use the VAP rubric for assessing her 
students' writing, and to teach her students to use the rubric to assess their own 
and their peers' writing. 

Conclusion 

The answer to the overall question of whether a single policy is likely to 
have a common effect on teachers with different beliefs, practices and contexts is, 
of course, "No." There were varying degrees and types of influences in these 
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four classrooms as a result of their first year of participation in the Vermont 
Assessment Program. These teachers began with different beliefs and practices, 
approached the VAP task differently, and finished in different places at the end 
of their first year. In each case, the influence was in relation to their existing 
beliefs, practices, and contexts. 

The influence of the VAP in specific classrooms ranged from none, to 
changes in organizational procedures, to students writing more, to students 
producing more varieties of writing, to students engaging in a new process that 
involved collecting their writing, reflecting on their own work and the work of 
others, and selecting from their collection. In some cases, the criteria of the VAP 
rubric helped teachers and students to look at writing differently, impacting the 
content and form of instruction and assessment. Although the changes that we 
observed and teachers reported may be smaller or different than anticipated by 
policy makers, it is important to understand that teachers do report a change. 
Indeed, three out of these four teachers reported changes in how they teach 
writing — and they were generally progressing in the desired direction. 

Analysis continues on the more complex interview and observational data 
related to these cases, but these basic findings are likely to hold. What are the 
implications for policy makers? Expectations for change need to be realistic. 
Teachers vary in their prerequisite experience and ability to move as quickly and 
directly as policy initiatives may assume. Change in teachers' instructional 
practices is usually gradual, evolutionary and context-dependent. This suggests 
the need for a long-range, flexible, multi-dimensional approach to professional 
development that is responsive to these differences. 
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